Sales system using apparel modeling system and method

ABSTRACT

A product modeling system and method are provided. On one embodiment, the product modeling system is used to model a piece of apparel, such as a shirt, with a design wherein the model with the design is used to display the piece of apparel with the design to a consumer.

PRIORITY CLAIMS/RELATED APPLICATIONS

This application is a continuation of and claims priority under 35 USC120 to U.S. patent application Ser. No. 11/925,714 filed on Oct. 26,2007 (to be issued as U.S. Pat. No. 11,157,977 on Oct. 26, 2021), theentirety of which is incorporated herein by reference.

APPENDICES

Appendix A (2 pages) contains an example of the pseudocode for finding aset of markers on a product;

Appendix B (1 page) contains an example of the code for remapping thecolor of an image using normalized ordinal color distribution;

Appendix C (4 pages) contains an example of the code for building acolor map in normalized histogram order with an index from a matchingcolor space;

Appendix D (2 pages) contains an example of the code for building alook-up table to remap the colors from a source sphere to a destinationsphere; and

Appendix E (3 pages) contains an example of the code for remapping thecolor of the source image with a source sphere color map to adestination image with the color map of the sphere color object.

Appendices A-E form part of the specification and are incorporatedherein by reference.

FIELD

The invention relates generally to a system and method for modeling apiece of apparel.

BACKGROUND

Electronic commerce (E-commerce) is a thriving business in which variousdifferent products and services are sold to a plurality of consumersusing an E-commerce site. The E-Commerce site may include a website thatallows a plurality of consumers to gain access to the website using anetwork, such as the Internet. The website may have a plurality of webpages wherein these web pages have images of a plurality of differentproducts that the consumer may purchase. The images contained in theplurality of web pages are two dimensional images. The website may alsoinclude a secure commerce portion that allows the consumer to select oneor more items, place those items in an electronic shopping cart and,when done shopping, check out and pay for the items that remain in theelectronic shopping cart using various payment services, such as PayPalor a credit card.

One limitation with these typical E-commerce systems is that the productavailable on the website, such as a shirt, may be modeled by a humanmodel to show the product and its design, but is shown to the consumeras a “flat” image since it is shown to the consumer on the display ofthe computer being used by the consumer. Thus, the actual design of theproduct and how the product looks in real life is often difficult todetermine from those images. This may result in consumers not purchasingthe product which is undesirable.

Another limitation of these typical E-commerce systems is that theproduct available on the website, such as a shirt, cannot be customizedby the consumer with a design on the product. Thus, the consumer cannotsee the customized product with the design and this also may result inconsumers not purchasing the product which is undesirable. Thus, it isdesirable to provide a system and method that provides better models forproducts and it is to this end that the system and method are directed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary sales system thatmay include an implementation of the product modeling system;

FIG. 2 illustrates an exemplary implementation of a product modelingmethod;

FIG. 3 illustrates further details of an exemplary implementation of aproduct modeling method;

FIGS. 4A and 4B illustrate further details of the process for marking aproduct in the product modeling method;

FIGS. 5A and 5B illustrate further details of the process for generatingimages of a product in the product modeling method;

FIGS. 6A-6D illustrate further details of the process for preprocessingthe model in the product modeling method;

FIGS. 7A-7C illustrate further details of the post processing process inthe product modeling method;

FIG. 8A illustrates an example of a design to be placed on a piece ofapparel;

FIG. 8B illustrates a typical image of a piece of apparel with thedesign in a typical system;

FIGS. 8C-8D illustrate the design on a piece of apparel in the productmodeling system;

FIG. 8E illustrates the process for placing the design on the model; and

FIGS. 9A-9C illustrate a process for changing the background againstwhich the piece of apparel with the design is displayed in the productmodeling system.

DETAILED DESCRIPTION OF ONE OR MORE EMBODIMENTS

The system and method are particularly applicable to a web basedclient/server model system and method for selling a shirt modeling witha product modeling system implemented in software on a computer and itis in this context that the system and method is illustrated anddescribed. It will be appreciated, however, that the system and methodcan be used for various products wherein the products may include otherapparel and other products in which it is desirable to provide bettersmodels of the products. For example, the system may be used for any typeof garment or piece of apparel, any item that can be worn or used by ahuman being or pet, such as a hat, backpack, dog sweater, etc. and/orany other product in which it is desirable to be able to display theproduct on a model. In addition, the system may be used with any productin which it is desirable to be able to display the product (with anirregular surface) with a design on it, such as a skateboard, a shoe. Inaddition, the system may be used to display a design on any item with anirregular surface, such as a wall, automobile body, a pencil and thelike. Furthermore, the system may be used to identify a product/item ina video wherein a design can be inserted into the product/item in thevideo. In addition, the system and method can be implemented in software(shown in the illustrated implementation), hardware or a combination ofhardware and software and may also be implemented on client/serversystem (shown in the illustrated implementation), a web server, aterminal, a peer to peer system and the like so that the system andmethod are not limited to the particular implementation of the system ormethod.

FIG. 1 is a diagram of a sales system 100 that markets and sellsproducts wherein the products sold by the system may be modeled using aproduct modeling system that generates more realistic models for theproducts. The sales system may include one or more client computingdevices 102, such as 102 a, 102 b, . . . , 102 n) wherein each computingdevice has at least one processing unit, memory, some persistent memory,some other memory, a display device and input/output devices (and eachmay be a personal computer, mobile device, cellular device, wirelessemail device, converged device such as a Treo or Blackberry) and thelike) that permit the user to interact with the client computing deviceas well as the sales system through a browser application being executedby the client computing device. Each client may establish a connectionwith and communicate over a network 104, such as the Internet using atypical secure or unsecure protocol with a sales unit 106 that may, inthis example, be one or more server computers. The sales unit 106 mayfurther comprise at least one web server 107 (implemented in hardware orsoftware or a combination of the two) that establishes a connection witheach client, receives requests from each client and communicates webpages with information and requests for client information to eachclient wherein the client interacts with the web server using a knownsecure or unsecure protocol and a typical browser application. The webserver, for example, may serve a web page that allows the consumer tobrowser the available products and designs and then, using the productmodeling system, model the particular design on a particular model of aparticular human model on a particular pose. The sales unit may furtherinclude a content store 108 that contains the product information andimages for the web pages and a well known ecommerce engine 110 that,once the consumer has selected a product with a particular design, mayallow the consumer to purchase the product with the particular design.

The sales unit 106 may also be coupled to a product modeling unit thatmay be implemented on the computing device with the typical elementswherein the product modeling unit may further comprise a productmodeling store 112, such as a software implemented database and a memoryin the product modeling unit may store a product modeling module 116that has a plurality of lines of computer code wherein the plurality oflines of computer code are executed by the processing unit to implementthe product modeling system and method as described below.

For purposes of illustrating the product modeling system and method, aproduct modeling method for a piece of apparel, such as a t-shirt, witha design is described below. However, the product modeling system mayalso be used for other products, such as other apparel and otherproducts in which it is desirable to provide betters models of theproducts. For example, the system may be used for any type of garment orpiece of apparel, any item that can be worn or used by a human being orpet, such as a hat, backpack, dog sweater, etc. and/or any other productin which it is desirable to be able to display the product on a model.In addition, the system may be used with any product in which it isdesirable to be able to display the product (with an irregular surface)with a design on it, such as a skateboard, a shoe. In addition, thesystem may be used to display a design on any item with an irregularsurface, such as a wall, automobile body, a pencil and the like.Furthermore, the system may be used to identify a product/item in avideo wherein a design can be inserted into the product/item in thevideo. The output of the product modeling method (an image of theproduct with a design shown on the product) may be used for variouspurposes. For example, the output may be used to generate a plurality ofproduct displays with designs on a website that allows consumers to seethe products. The example described below is a system in which theproduct modeling system is tied to a product marketing and sellingcompany wherein the product marketing and selling company has control ofmodels and images of the product modeling system. In anotherimplementation/embodiment of the product modeling system, the system maypermit a consumer to provide their own images/models, such as models ofthe actual consumer, so that the consumer can upload the image to aservice and then have the selected design displayed on the model of theactual consumer wherein the service provides: 1) the model components(to create the model form); 2) a tool to upload/modify the model imagesto the service; and 3) a tool to display the model with the design tothe consumer.

FIG. 2 illustrates an exemplary implementation of a product modelingmethod 120 that displays a model with a design on the model wherein themodel is a realistic representation of a person with a piece of apparelthat has the design on the piece of apparel. The methods shown in FIGS.2 and 3, the processes described below may be performed by the productmodeling module 116 described above. A consumer may select a design(122) such as the design shown in FIG. 8A and a warp process (124) maybe performed to generate a warp design (128). The consumer may alsoselect a background (126) for the model such as the backgrounds shown inFIGS. 9A-9C. Once the background and design are chosen by the consumer,the design is warped and then surface shading (130) and a surfacespecular process (132) is performed. Once these processes are completed,the model is created with the design (134) wherein the model with thedesign is shown to the consumer. In one embodiment, the model with thedesign is displayed to the consumer to assist the consumer in previewingthe product with the design before the consumer purchases the productwith the design, such as through an E-commerce website. Now, the productmodeling method is described in more detail.

FIG. 3 illustrates further details of an exemplary implementation of aproduct modeling method 140 when used with a piece of apparel. Theproduct modeling method (and the processes set forth below) areimplemented, in one embodiment and implementation, as a plurality oflines of computer code that are part of the product modeling module thatare executed by a processing unit 106 that is part of the productmodeling system. In the method, a piece of apparel is created with aplurality of markers (142) that are used to capture information aboutthe piece of apparel when the piece of apparel is worn by a human model.The plurality of markers may be a marker pattern that encodes, in twodimensions, a flexible substrate that may be detected when the flexiblesubstrate is placed on a complex three dimensional surface wherein thecoverage area of the marker pattern does not substantially occlude thesubstrate that it encodes. For example, the plurality of markers maycover a predetermined percentage, such as 50%, of the piece of apparel,that allow the system to capture information about the piece of apparelwhen the piece of apparel is worn by a human model. In oneimplementation, the plurality of markers may form a grid. In moredetail, the markers that form a grid on a flat surface (the piece ofapparel flat on a surface when the markers are properly positioned onthe piece of apparel) may be used to map to a grid of markers on anon-flat surface (the piece of apparel when worn on a human model). Asshown in FIG. 4A, the grid of markers 186 on the flat surface are mappedto a grid 187 with the same markers in the same positions on a non-flatsurface so that the mapping between the grid on the flat surface and thegrid on the non-flat surface is determined. The system may interpolatethe marker locations to generate a mapping from the plurality of markersto the grid on the flat surface and may then store the mapping to avoidrecalculation of the mapping each time. In one embodiment, the markersmay be a number of non-visible lines that form a grid. In anotherembodiment, the markers may be a plurality of optical markers 190 thatmay be affixed to a piece of apparel 192 as shown in FIG. 4B thatpermits the optical tagging of the piece of apparel to map the surfaceof the piece of apparel when worn by a human model. The optical markersmay be made of a reflective material, a colorized material or adiffraction pattern. The reflective material may be retro-reflectivematerial. The colorized material may be pigmented material. The markersmay have various shapes (including the dot shape shown in FIG. 4B) andsizes and the method is not limited to any particular shape of themarkers. In one embodiment, the plurality of markers may be a filmmaterial that has the retro-reflective material in a particular shape.In yet another embodiment, the markers may be a set of markers that forma grid wherein the markers are placed onto the piece of apparelelectronically or by other means. In one embodiment in which the productmodeling system is used by a business entity that sells apparel, eachpiece of apparel is placed onto a plurality of human models of differentshapes and/or sizes (as shown in FIGS. 5A and 5B) so that the consumercan then choose a model for the piece of apparel that is closest to theintended wearer of the piece of apparel. In another embodiment in whicheach consumer may create his own model for a piece of apparel, theconsumer is provided with the markers (either electronically or asphysical markers) so that the consumer can affix the markers to a pieceof apparel and then performs the other processes described below. In yetanother embodiment, the product modeling system may allow a plurality ofusers (such as a community of users) to generate a plurality of modelsthat may then be uploaded to the product modeling system.

Once the one or more pieces of apparel are prepared with the markers, animage for each piece of apparel on each different human model may begenerated (150) such as by using a camera to take a picture of the pieceof apparel being worn by a human model. Prior to taking the image of thepiece of apparel with the markers on the human model, the lighting fortaking the image is determined. When the user/consumer generates themodels, the product modeling system may provide instructions for takingan image of the piece of apparel such as using a flash, using aparticular exposure, etc. . . . . In one implementation of the productmodeling system, the product modeling system may download a piece ofcode directly to a user/consumer's camera, such as a digital camera, toset up the camera properly to take the image of the product or item. Inparticular, the surface model and illumination model for each piece ofapparel is determined which also allows the color and lighting for theimage to be accurately determined.

Once the image of the piece of apparel on a plurality of human models ina plurality of different poses are taken, the model for the piece ofapparel on a particular model in a particular pose are preprocessed(160) by the product modeling system. During the preprocessing, theproduct modeling system may detect the plurality of markers on the pieceof apparel image, remove the marker images from the image of the pieceof apparel and then generate a representation of the surface of thepiece of apparel when worn by the human model.

In one implementation, the markers may be detected by a distinguishingfeature of the markers (spectral difference, reflective difference,textual difference and/or temporal difference), refined by matchinggeometric properties of the pattern (local pattern finding) andreconstructed by matching the known pattern (local patterns assembledinto a known complete pattern.) The reconstructed pattern may then beused to model the shape of the flexible substrate. The product modelingsystem may have a plurality of local samples of the original unmarkedsubstrate so that the marker pattern can be replaced using the texturesof the unmarked substrate as an example that yields an unmarked imagesuitable for commercial display.

The preprocessing process is shown in FIGS. 6A-6C with FIG. 6Aillustrating the image of the piece of apparel with the markers, FIG. 6Billustrating the plurality of markers identified on the piece of appareland FIG. 6C illustrates the image of the piece of apparel with themarkers removed. Appendix A (2 pages), incorporated herein by reference,contains an example of the pseudocode for identifying the markers on aproduct in one implementation of the product modeling system. The stepsof the marker identification process for one implementation are setforth in Appendix A. In one implementation, the markers are detected byvisible detection. In another implementation of the system, the markersmay be detected by a temporal process in which infrared radiation may beused to image the markers at several different times and then thepattern of the markers is detected based on the images of the markers atseveral different times.

During the identification of the markers, the product modeling systemmay use various techniques. For example, edge detection may be used toidentify each marker and the spacing between the markers that can thenbe used to generate the grid of markers on the surface of the piece ofapparel when worn on a human model that thus allows the surface of thatpiece of apparel on the particular human model in a particular pose tobe accurately determined. Alternatively, the system may threshold at thewhite color based on the color calibration and then locate elementsabove the threshold and then also identify the background includingelements of the human model such as jewelry, an eye or the backgroundbehind the human model. The system may also use histograms to identifythe markers and the background.

The marker images (once identified) may be removed from the image of thepiece of apparel (as shown in FIG. 6C) by various processes. Forexample, the markers may be removed by, for each marker location,identifying the texture adjacent the marker and then filling in thelocation of the marker with the texture in the adjacent area.Alternatively, the system may use image coherence and synthesize theimage to remove the markers in the image.

To generate the representation of the contours of the surface of thepiece of apparel when worn by a particular human model in a particularpose, the system maps the position of the markers 190 relative to eachother as shown in FIG. 6D into a set of contour curves 194 thatrepresent the surface of the piece of apparel when worn by a particularhuman model in a particular pose. Since the system has information aboutthe markers and the grid that they form on a flat surface as shown FIG.4A, the system is able to determine the contours of the surface of thepiece of apparel when worn by a particular human model in a particularpose.

Once the contours of the surface is determined and the preprocessing iscompleted, the model of the piece of apparel when worn by a particularhuman model in a particular pose may be retouched (162) as needed. Then,the model is post-processed (170) by the product model system. Duringthe post-processing process, the product model system colorizes themodel using a color mapping module that is part of the product modelsystem. The colorizing allows each model for each piece of apparel on aparticular human model in a particular pose to have the piece of apparelconverted into any colors such as the two different colors shown inFIGS. 7A and 7B. As shown in FIG. 7C, the system may use the colorcalibration card with a known spectral response for each session tocalibrate images for the same session. To change the color for the pieceof apparel, the fabric may be wrapped onto a sphere as shown in FIG. 7Cwhich is then mapped to the model to change the color of the model.

Appendices B-E, incorporated herein by reference, illustrate, for aparticular implementation of the product modeling system, the codefor 1) remapping the color of an image using normalized ordinal colordistribution; 2) building a color map in normalized histogram order withan index from a matching color space; 3) building a look-up table toremap the colors from a source sphere to a destination sphere; and 4)remapping the color of the source image with a source sphere color mapto a destination image with the color map of the sphere color object.Using the code set forth in these appendices (and the process stepsdescribed in these appendices), the color mapping process: 1) builds acolor map (the BuildMap code in Appendix C) for the source image using asphere to build a histogram and then a sorted table; 2) builds a remaptable (the BuildReMap table code in Appendix D); 3) remaps the imagecolors (the code in Appendices B and E) onto the product. The system mayalso layer color and texture so that the colorized model of theparticular piece of apparel on the particular human model in theparticular pose more accurately emulates different fabrics and/orthreads of the fabric which results, for example, in an accurateemulation of the printed ink of the design on the piece of apparel withthe particular type of fabric.

Once the colorization is completed, the model for a particular piece ofapparel on a particular human model in a particular pose is integratedinto a service (180) such as a website that has the pieces of apparelwith particular designs for sale to consumers.

When the model is integrated into the service, the product modelingsystem may perform warp mapping (182) on a design selected by theconsumer and permit the user to select a particular background (184). Anexample design is shown in FIG. 8A. The exemplary design shown on apiece of apparel in a typical system with a flat image is shown in FIG.8B. Using the product modeling system, a mapping between the designimage and the surface contour of the model of the particular piece ofapparel on the particular human model in the particular pose (See forexample FIG. 8D) is done so that the design is shown on the model asshown in FIG. 8E is a more realistic three dimensional manner.

During the warp mapping (that may be a bicubic image warp), a grid ofthe design 200 is mapped to the surface contour grid 202 which is thenplaced onto the piece of apparel to generate the more realistic modelfor the piece of apparel with the design as shown in FIG. 8D. In themapping process, a point in the design is mapped to the surface contourgrid which is in turn mapped onto the piece of apparel. The imagebackground can be easily exchanged by the product modeling system asshown in FIGS. 9A-9C.

Although the example provided herein is for a piece of apparel (a shirt)worn by a human being, the product modeling system may be used forvarious different products (other pieces of apparel, other garments,hats, shoes, pet clothing, inanimate objects such as cups) with variousdifferent models (human models, animal models, inanimate models such asrobots or mannequins) and with any number of different poses for themodels since the above example is merely illustrative.

While the foregoing has been with reference to a particular embodimentof the invention, it will be appreciated by those skilled in the artthat changes in this embodiment may be made without departing from theprinciples and spirit of the invention, the scope of which is defined bythe appended claims.

1. A system for selling products modeled using a product modelingsystem, comprising: a computer based sales unit for making a pluralityof products available for sale; a plurality of markers that form amarker pattern on a product that does not occlude a surface of theproduct; a computer based product modeling unit having a processor thatexecutes a plurality of lines of instructions that configure theprocessor to: capture an image of the project on an object and theplurality of markers; detect, based on the captured image, each of theplurality of markers on the product to determine a contour of theproduct and a contour of a design area on the product; electronicallyapply a user design to the product; and generate a visual representationof the user design on the product when the product having the userdesign is on the object using the captured lighting, texture andcontours of the product and the object so that the visual representationof the design on the product has the contour of the surface of theproduct; and an ecommerce system that permits the consumer to purchasethe product with the design on the product that was modeled by theproduct modeling unit.
 2. The system of claim 1, wherein the processoris further configured to detect a distinguishing feature of the markerto detect the market on the product.
 3. The system of claim 2, whereinthe distinguishing feature is one of a spectral difference, a reflectivedifference, a textual difference and a temporal difference.
 4. Thesystem of claim 1, wherein the computer based sales unit furthercomprises a web server that generates a web page that displays thevisual representation of the product with the design on a web page to aconsumer.
 5. The system of claim 4, wherein the object further comprisesa human model, a mannequin or an animal.
 6. The system of claim 4,wherein each marker further comprises a piece of pigment, a piece ofreflective material or a piece of retro-reflective material.
 7. Thesystem of claim 4, wherein the plurality of markers further comprise agrid of lines not visible to a human on the product.
 8. The system ofclaim 4, wherein the product further comprises a piece of apparel, agarment, an item worn by a human being or an item worn by an animal. 9.A method for sales using product modeling, the method comprising:providing a plurality of products available for sale; capturing an imageof the project on an object and the plurality of markers; detecting,based on the captured image, each of the plurality of markers on theproduct to determine a contour of the product and a contour of a designarea on the product; electronically applying a user design to theproduct; generating a visual representation of the user design on theproduct when the product having the user design is on the object usingthe captured lighting, texture and contours of the product and theobject so that the visual representation of the design on the producthas the contour of the surface of the product; displaying the visualrepresentation to a consumer of a design on a product when on an objectusing the captured contour of a surface of the product on the object;and permitting a consumer to purchase the product with the design on theproduct that was modeled by the product modeling unit.
 10. The method ofclaim 9, wherein detecting the markers further comprises detecting adistinguishing feature of the marker to detect the market on theproduct.
 11. The method of claim 10, wherein the distinguishing featureis one of a spectral difference, a reflective difference, a textualdifference and a temporal difference.
 12. The method of claim 9 furthercomprising generating a web page that displays the visual representationof the product with the design on a web page to the consumer.
 13. Themethod of claim 12, wherein the object further comprises a human model,a mannequin or an animal.
 14. The method of claim 12, wherein eachmarker further comprises a piece of pigment, a piece of reflectivematerial or a piece of retro-reflective material.
 15. The method ofclaim 12, wherein the plurality of markers further comprise a grid oflines not visible to a human on the product.
 16. The method of claim 12,wherein the product further comprises a piece of apparel, a garment, anitem worn by a human being or an item worn by an animal.